Skip to content

feat(signals): wire test coverage classification across contributor surfaces - #2680

Closed
luciferlive112116 wants to merge 1 commit into
JSONbored:mainfrom
luciferlive112116:feat/signals-test-coverage-classification
Closed

feat(signals): wire test coverage classification across contributor surfaces#2680
luciferlive112116 wants to merge 1 commit into
JSONbored:mainfrom
luciferlive112116:feat/signals-test-coverage-classification

Conversation

@luciferlive112116

@luciferlive112116 luciferlive112116 commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Extend test-evidence with fixture/mock path detection and a shared buildTestCoverageSummary() (strong / adequate / weak / absent).
  • Surface testCoverage on local workspace intelligence and add a new weak_test_coverage slop signal (info-level, weight 10 — does not block alone).
  • Classify open PRs with disproportionately weak test coverage separately from fully missing tests in the contributor open-PR monitor.
  • Update OpenAPI schema and generated openapi.json.

Scope

  • Conventional Commit feature title (feat(signals): …).
  • Focused signals-layer feature with regression tests only.
  • No linked issue — repo linkedIssuePolicy is preferred; self-evident contributor-intelligence feature.

Validation

  • git diff --check
  • npm run typecheck
  • Targeted unit tests (203 passing across signals/local-branch/openapi suites)
  • npm run ui:openapi (OpenAPI contract updated)
  • Full npm run test:ci — CI on GitHub Actions

Safety

  • Deterministic, public-safe signals only; no auth/API/wallet changes.
  • Empty PR file lists still treated as unknown (no false missing_tests).

@luciferlive112116 @cursoragent
feat(signals): wire test coverage classification across contributor s…
3b626d7 
…urfaces

Extend test-evidence with fixture-path detection and a shared coverage summary, then surface weak/adequate/strong classification in local workspace intelligence, slop assessments, the contributor open-PR monitor, and the OpenAPI contract.

Co-authored-by: Cursor <cursoragent@cursor.com>
@dosubot dosubot Bot added the size:L label Jul 3, 2026
@codecov

codecov Bot commented Jul 3, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.82759% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.10%. Comparing base (b3244d1) to head (3b626d7).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
src/signals/slop.ts 87.50% 0 Missing and 2 partials ⚠️
src/signals/test-evidence.ts 96.55% 0 Missing and 1 partial ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #2680      +/-   ##
==========================================
- Coverage   96.10%   96.10%   -0.01%     
==========================================
  Files         237      237              
  Lines       26542    26585      +43     
  Branches     9625     9640      +15     
==========================================
+ Hits        25509    25549      +40     
  Misses        424      424              
- Partials      609      612       +3
Files with missing lines Coverage Δ
src/openapi/schemas.ts 100.00% <ø> (ø)
src/signals/contributor-open-pr-monitor.ts 96.36% <100.00%> (+0.13%) ⬆️
src/signals/local-workspace-intelligence.ts 100.00% <100.00%> (ø)
src/signals/test-evidence.ts 96.87% <96.55%> (-3.13%) ⬇️
src/signals/slop.ts 97.07% <87.50%> (-1.01%) ⬇️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb

loopover-orb Bot commented Jul 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Caution

🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥

🛑 Gittensory review result - reject/close recommended

Review updated: 2026-07-03 05:20:10 UTC

10 files · 1 AI reviewer · 1 blocker · readiness 80/100 · CI failing · blocked

🛑 Suggested Action - Reject/Close

  • AI reviewers agree on a likely critical defect: src/signals/test-evidence.ts:45 computes `testToChangedRatio` as evidence paths divided by all changed paths, so a reachable PR with one source file, one test file, and many docs/config files is incorrectly classified as `weak` instead of measuring coverage against the source/evidence surface. — Resolve the flagged defect, or override if the AI reviewers are mistaken, then re-run the gate.

Review summary
The change correctly centralizes test/fixture evidence classification and wires the new weak coverage category through slop, local intelligence, contributor monitoring, and OpenAPI. The main correctness issue is that the shared ratio uses every changed path as the denominator, so unrelated non-source files can incorrectly downgrade a source change with matching tests into weak coverage. The fixtures-as-evidence behavior and empty file-list handling are otherwise coherent in the visible diff.

Blockers

  • src/signals/test-evidence.ts:45 computes `testToChangedRatio` as evidence paths divided by all changed paths, so a reachable PR with one source file, one test file, and many docs/config files is incorrectly classified as `weak` instead of measuring coverage against the source/evidence surface.
Nits — 5 non-blocking
  • src/signals/contributor-open-pr-monitor.ts:90 calls `missingTestsFromFiles(files)` twice for the same PR; compute the value once next to `weakTestCoverageFromFiles(files)` to avoid drift and repeated classification work.
  • test/unit/test-evidence.test.ts should add a mixed source+test+docs/config case so the intended denominator is locked down and this regression cannot recur.
  • In `src/signals/test-evidence.ts`, base the ratio on `sourcePathCount + evidencePathCount` rather than `changedPathCount`, while still reporting `changedPathCount` for diagnostics.
  • Add contributor monitor coverage for a PR that includes source, matching tests, and unrelated non-code files to prove the monitor does not misclassify it as `weak_test_coverage`.
  • Reuse the computed missing/weak coverage booleans in `buildContributorOpenPrMonitor` before calling both `mapPendingClassToWorkClassification` and `buildNextStepPacket`.

Why this is blocked

  • src/signals/test-evidence.ts:45 computes `testToChangedRatio` as evidence paths divided by all changed paths, so a reachable PR with one source file, one test file, and many docs/config files is incorrectly classified as `weak` instead of measuring coverage against the source/evidence surface.

CI checks failing

  • codecov/patch — 94.82% of diff hit (target 99.00%)
Signal Result Evidence
Code review ❌ 1 blocker 1 reviewer
Linked issue ✅ No-issue rationale PR body explains why no issue is linked.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (size label size:L; no linked issue context).
Validation posture ❌ 5/25 Preflight is holding this PR: the review lane is unavailable, so it is not ready for automated review.
Contributor workload ✅ 10/10 Author activity: 43 registered-repo PR(s), 19 merged, 7 issue(s).
Contributor context ✅ Confirmed Gittensor contributor luciferlive112116; Gittensor profile; 43 PR(s), 7 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Review context
  • Author: luciferlive112116
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: not available
  • Official Gittensor activity: 43 PR(s), 7 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Await review-lane availability.
  • Refresh registry data or choose a registered active repo.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added gittensor gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. labels Jul 3, 2026
JSONbored
JSONbored reviewed Jul 3, 2026
View reviewed changes

@JSONbored JSONbored left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge conflicts + review blockers.

This branch has conflicts that must be resolved
Use the [web editor](https://github.com/JSONbored/gittensory/pull/2680/conflicts) or the command line to resolve conflicts before continuing.

test/unit/test-evidence.test.ts

@JSONbored JSONbored closed this Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JSONbored JSONbored JSONbored left review comments

Assignees

@luciferlive112116 luciferlive112116

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@luciferlive112116 @JSONbored